Revamp issue-labeler project by MaiLinhP · Pull Request #13310 · Azure/azure-sdk-tools

MaiLinhP · 2025-12-10T22:40:37Z

Added an `AzureSdkIssueGeneratorService` Function App that will reference azure-sdk-for-net issues to generate synthetic issues for other repos and add TSV file to blob storage for model training if needed.

Replaced all old ML labeler processes with streamlined `IssueLabelerMLPipeline`. Most of this pipeline is meant to be run locally for more control over model training processes. It includes:

Downloader: Download applicable issues to local .tsv file.
Trainer: Use downloaded issues (along with synthetic issues generated from AzureSdkIssueGeneratorService if available) to train Category and Service models. This will also provide per label statistics as well as a list of labels that need more data for training. This list can be passed back into AzureSdkIssueGeneratorService to generate issues with specific labels.
Predictor: Use the category and service model zip files produced from Trainer to predict issues.
Tester: Test models on all labeled issues for # of matches, mismatches, and no predictions. Also provides statistics on how models perform.
DevOpsModelRetrainer: Process to be handed to DevOps team to periodically retrain models automatically. This download issues, train models along with synthetic data if any, then replace current models with new ones.

Unified config setup.

Previously, most function app configs use {org}/{repo}:{key} or defaults:{key}. However, as ModelHolderFactory is a singleton and didn't have per repo config at runtime, we were doing roundabout logics to configure model files (check old LegacyLabeler and IModelHolderFactoryLite.cs for details). Per repo config is now available to ModelHolderFactory, which simplifies many configs and cut out unnecessary name translation/extraction logic.

Simplified the flow from `AzureSdkIssueLabelerService` to `Labeler`.

Previously, there were many layers - ModelHolder, ModelHolderFactory, ModelHolderFactoryLite, Labeler, LabelerLite, LegacyLabeler. Replaced LegacyLabeler with MLLabeler, simplified the rest down to ModelHolder and ModelHolderFactory.

Previously, ModelHolder was also unaware of which label type it is, and the ordering of service vs category models in ModelHolderFactory was dictated by order listed in defaults:IssueModel.{repo}.BlobConfigNames. This made code unreadable without constantly referencing app config. ModelHolder now has modelType which makes ordering irrelevant, simplifies call to config and making code much more readable.

Cleaned up codebase

Deleted all unused references and refactored functions.

…works with repo specific Configuration

…es to blob storage

microsoft-github-policy-service · 2025-12-10T22:40:47Z

@MaiLinhP please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

Contribution License Agreement

This Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
and conveys certain license rights to Microsoft Corporation and its affiliates (“Microsoft”) for Your
contributions to Microsoft open source projects. This Agreement is effective as of the latest signature
date below.

Definitions.
“Code” means the computer software code, whether in human-readable or machine-executable form,
that is delivered by You to Microsoft under this Agreement.
“Project” means any of the projects owned or managed by Microsoft and offered under a license
approved by the Open Source Initiative (www.opensource.org).
“Submit” is the act of uploading, submitting, transmitting, or distributing code or other content to any
Project, including but not limited to communication on electronic mailing lists, source code control
systems, and issue tracking systems that are managed by, or on behalf of, the Project for the purpose of
discussing and improving that Project, but excluding communication that is conspicuously marked or
otherwise designated in writing by You as “Not a Submission.”
“Submission” means the Code and any other copyrightable material Submitted by You, including any
associated comments and documentation.
Your Submission. You must agree to the terms of this Agreement before making a Submission to any
Project. This Agreement covers any and all Submissions that You, now or in the future (except as
described in Section 4 below), Submit to any Project.
Originality of Work. You represent that each of Your Submissions is entirely Your original work.
Should You wish to Submit materials that are not Your original work, You may Submit them separately
to the Project if You (a) retain all copyright and license information that was in the materials as You
received them, (b) in the description accompanying Your Submission, include the phrase “Submission
containing materials of a third party:” followed by the names of the third party and any licenses or other
restrictions of which You are aware, and (c) follow any other instructions in the Project’s written
guidelines concerning Submissions.
Your Employer. References to “employer” in this Agreement include Your employer or anyone else
for whom You are acting in making Your Submission, e.g. as a contractor, vendor, or agent. If Your
Submission is made in the course of Your work for an employer or Your employer has intellectual
property rights in Your Submission by contract or applicable law, You must secure permission from Your
employer to make the Submission before signing this Agreement. In that case, the term “You” in this
Agreement will refer to You and the employer collectively. If You change employers in the future and
desire to Submit additional Submissions for the new employer, then You agree to sign a new Agreement
and secure permission from the new employer before Submitting those Submissions.
Licenses.

Copyright License. You grant Microsoft, and those who receive the Submission directly or
indirectly from Microsoft, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license in the
Submission to reproduce, prepare derivative works of, publicly display, publicly perform, and distribute
the Submission and such derivative works, and to sublicense any or all of the foregoing rights to third
parties.
Patent License. You grant Microsoft, and those who receive the Submission directly or
indirectly from Microsoft, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license under
Your patent claims that are necessarily infringed by the Submission or the combination of the
Submission with the Project to which it was Submitted to make, have made, use, offer to sell, sell and
import or otherwise dispose of the Submission alone or with the Project.
Other Rights Reserved. Each party reserves all rights not expressly granted in this Agreement.
No additional licenses or rights whatsoever (including, without limitation, any implied licenses) are
granted by implication, exhaustion, estoppel or otherwise.

Representations and Warranties. You represent that You are legally entitled to grant the above
licenses. You represent that each of Your Submissions is entirely Your original work (except as You may
have disclosed under Section 3). You represent that You have secured permission from Your employer to
make the Submission in cases where Your Submission is made in the course of Your work for Your
employer or Your employer has intellectual property rights in Your Submission by contract or applicable
law. If You are signing this Agreement on behalf of Your employer, You represent and warrant that You
have the necessary authority to bind the listed employer to the obligations contained in this Agreement.
You are not expected to provide support for Your Submission, unless You choose to do so. UNLESS
REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING, AND EXCEPT FOR THE WARRANTIES
EXPRESSLY STATED IN SECTIONS 3, 4, AND 6, THE SUBMISSION PROVIDED UNDER THIS AGREEMENT IS
PROVIDED WITHOUT WARRANTY OF ANY KIND, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTY OF
NONINFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
Notice to Microsoft. You agree to notify Microsoft in writing of any facts or circumstances of which
You later become aware that would make Your representations in this Agreement inaccurate in any
respect.
Information about Submissions. You agree that contributions to Projects and information about
contributions may be maintained indefinitely and disclosed publicly, including Your name and other
information that You submit with Your Submission.
Governing Law/Jurisdiction. This Agreement is governed by the laws of the State of Washington, and
the parties consent to exclusive jurisdiction and venue in the federal courts sitting in King County,
Washington, unless no federal subject matter jurisdiction exists, in which case the parties consent to
exclusive jurisdiction and venue in the Superior Court of King County, Washington. The parties waive all
defenses of lack of personal jurisdiction and forum non-conveniens.
Entire Agreement/Assignment. This Agreement is the entire agreement between the parties, and
supersedes any and all prior agreements, understandings or communications, written or oral, between
the parties relating to the subject matter hereof. This Agreement may be assigned by Microsoft.

Copilot

Pull request overview

This PR represents a significant architectural refactoring of the issue-labeler project, replacing legacy ML labeling infrastructure with a modern streamlined pipeline approach.

Key Changes

New Synthetic Issue Generation: Added AzureSdkIssueGeneratorService Function App to generate synthetic issues from azure-sdk-for-net issues for improved model training
Streamlined ML Pipeline: Replaced old ML labeler processes with IssueLabelerMLPipeline containing local tools for downloading, training, testing, and predicting issue labels
Simplified Architecture: Removed Hubbup.MikLabelModel project and replaced LegacyLabeler with MLLabeler, eliminating multiple abstraction layers and simplifying the codebase
Unified Configuration: Consolidated config setup with per-repository support in ModelHolderFactory, removing complex name translation logic

Reviewed changes

Copilot reviewed 123 out of 123 changed files in this pull request and generated 28 comments.

Show a summary per file

File	Description
`IssueLabelerService/Labelers/MLLabeler.cs`	New ML labeler replacing LegacyLabeler with simplified prediction logic
`IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs`	New issue generator service for creating synthetic training data
`IssueLabelerMLPipeline/src/`	Complete new ML pipeline with Downloader, Trainer, Predictor, Tester, and DevOpsModelRetrainer tools
`IssueLabeler.Shared/ModelHolder.cs`	Refactored to include modelType awareness and simplified blob loading
`IssueLabeler.Shared/IModelHolderFactory.cs`	New interface with per-repository configuration support
`Hubbup.MikLabelModel/`	Entire project and dependencies removed
`tests/Hubbup.MikLabelModel.Tests/`	Test project removed as part of cleanup

Comments suppressed due to low confidence (2)

tools/issue-labeler/src/IssueLabelerService/AzureSdkIssueGeneratorService.cs:46

This assignment to result is useless, since its value is never read.
tools/issue-labeler/src/IssueLabelerService/AzureSdkIssueGeneratorService.cs:51
This assignment to answer is useless, since its value is never read.

Copilot · 2025-12-10T22:50:43Z

tools/issue-labeler/src/IssueLabelerService/AzureSdkIssueGeneratorService.cs

+            }
+
+            var config = _configurationService.GetDefault();
+            TriageOutput result = new TriageOutput {};


Unused variable 'result' is created but never used. The return statement from the issue generator service is assigned to 'answer' (line 51), but then the function returns without using either variable. This appears to be incomplete implementation or should return a proper response.

Copilot · 2025-12-10T22:50:43Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Common/LabelType.cs

+public enum LabelType
+{
+    Category = 1,
+    Service = 2
+}


Missing copyright and license header. For consistency with other files in the project, add the standard copyright header at the top of this file.

+1 . We do need that.

Copilot · 2025-12-10T22:50:43Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+                public static string FormatTemplate(string template, Dictionary<string, string> replacements, ILogger logger)
+        {
+            if (string.IsNullOrEmpty(template))
+                return string.Empty;
+
+            string result = template;
+
+            foreach (var replacement in replacements)
+            {
+                if (!result.Contains($"{{{replacement.Key}}}"))
+                {
+                    logger.LogWarning($"Replacement value for {replacement.Key} does not exist in {template}.");
+                }
+                result = result.Replace($"{{{replacement.Key}}}", replacement.Value);
+            }
+
+            // Replace escaped newlines with actual newlines
+            return result.Replace("\\n", "\n");
+        }


The method 'FormatTemplate' appears to be a duplicate of similar functionality in 'AzureSdkIssueLabelerService' class (line 67 references it). Consider extracting this method to a shared utility class to avoid code duplication and improve maintainability.

Copilot · 2025-12-10T22:50:43Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+                foreach (var issue in issues)
+                {
+                    if (issue.Category is null || issue.Service is null || issue.Title is null || issue.Body is null
+                    || !applicableCategoryLabels.Contains(issue.Category) || !applicableServiceLabels.Contains(issue.Service)) continue;
+
+                    await writer.WriteLineAsync(FormatIssueRecord(issue.Category, issue.Service, issue.Title, issue.Body));
+                }


This foreach loop implicitly filters its target sequence - consider filtering the sequence explicitly using '.Where(...)'.

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

Copilot · 2025-12-10T22:50:51Z

tools/issue-labeler/src/IssueLabelerService/Labelers/LabelerFactory.cs

-        private IModelHolderFactoryLite _modelHolderFactory;
        private ILogger<LabelerFactory> _logger;
-        private ILabelerLite _labeler;
+        private IModelHolderFactory _modelHolderFactory;


Field '_modelHolderFactory' can be 'readonly'.

Copilot · 2025-12-10T22:50:52Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+{
+    public class OpenAiIssueGenerator : IIssueGeneratorService
+    {
+        private RepositoryConfiguration _config;


Field '_config' can be 'readonly'.

Copilot · 2025-12-10T22:50:52Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+    public class OpenAiIssueGenerator : IIssueGeneratorService
+    {
+        private RepositoryConfiguration _config;
+        private TriageRag _ragService;


Field '_ragService' can be 'readonly'.

Copilot · 2025-12-10T22:50:52Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+    {
+        private RepositoryConfiguration _config;
+        private TriageRag _ragService;
+        private ILogger<IssueGeneratorFactory> _logger;


Field '_logger' can be 'readonly'.

Copilot · 2025-12-10T22:50:52Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+        private TriageRag _ragService;
+        private ILogger<IssueGeneratorFactory> _logger;
+        private readonly KnowledgeAgentRetrievalClient _retrievalClient;
+        private BlobServiceClient _blobClient;


Field '_blobClient' can be 'readonly'.

jeo02

Looks good

Wish I could have figured out GraphQL myself, but it was quite difficult to use. Do you encounter less rate limiting through GraphQL vs using the API through octokit? I imagine it should be quite an improvement.

Mostly just nits on the styling. I have nothing against top-level statements (even though the constant comments made it seem so) but since everything else is in classes I think we should keep it consistent.

Also saw a couple of classes that are declared all in one file. To find it easier it would be better to just declare them separately under Models folder.

The synthetic issues are really cool !!

jeo02 · 2026-01-07T23:26:18Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/DevOpsModelRetrainer/DevOpsModelRetrainer.cs

+using Azure.Identity;
+using Azure.Storage.Blobs;
+using Microsoft.Extensions.Configuration;


Any reason this class is all top level rather than being a class?

jeo02 · 2026-01-07T23:28:52Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Downloader/Downloader.cs

+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.


Also all top level would it not be better to have it be a class?

jeo02 · 2026-01-07T23:31:45Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/GitHubClient/GitHubApi.cs

+    /// <returns>The downloaded issues as an async enumerable collection of tuples containing the issue and its predicate-matched label (when only one matcing label is found).</returns>
+    public static async IAsyncEnumerable<(Issue Issue, string CategoryLabel, string ServiceLabel)> DownloadIssues(
+        string githubToken,
+        string org, string repo,


(nit)

Suggested change

string org, string repo,

string org,

string repo,

tools/issue-labeler/src/IssueLabelerMLPipeline/src/GitHubClient/GitHubApi.cs

jeo02 · 2026-01-07T23:40:52Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/GitHubClient/GitHubClient.csproj

+    <PackageReference Include="GitHubJwt" />
+    <PackageReference Include="GraphQL.Client" />
+    <PackageReference Include="GraphQL.Client.Serializer.SystemTextJson" />
+    <PackageReference Include="Octokit" />


won't need octokit here either

jeo02 · 2026-01-07T23:46:59Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Predictor/Models.cs

@@ -0,0 +1,55 @@
+// Licensed to the .NET Foundation under one or more agreements.


Class per file :) like the other comment

jeo02 · 2026-01-07T23:47:48Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Predictor/Predictor.cs

@@ -0,0 +1,193 @@
+// Licensed to the .NET Foundation under one or more agreements.


top level but seeing as so much of the code is structured by classes, I think we should keep consistent and make this a class as well. (Unless its top level for some reason I don't see)

jeo02 · 2026-01-07T23:48:36Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Tester/Tester.cs

@@ -0,0 +1,236 @@
+// Licensed to the .NET Foundation under one or more agreements.


jeo02 · 2026-01-07T23:51:19Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Tester/Tester.cs

+    return bestScore is not null ? (bestScore.Label, bestScore.Score) : ((string?)null, (float?)null);
+}
+
+class TestStats


Put this in its own file

jeo02 · 2026-01-07T23:52:25Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Trainer/Trainer.cs

@@ -0,0 +1,43 @@
+// Licensed to the .NET Foundation under one or more agreements.


jsquire · 2026-01-09T20:18:39Z

tools/issue-labeler/Directory.Build.props

    <MicrosoftNETTestSdkVersion>16.11.0</MicrosoftNETTestSdkVersion>
  </PropertyGroup>

+  <PropertyGroup>


Is this for local support of the Azure Function? I'm wondering if we should move this to a local props file that we ignore so that CI doesn't get confused.

jsquire · 2026-01-09T20:22:17Z

tools/issue-labeler/src/IssueLabeler.Shared/IModelHolderFactory.cs

+        Task<IModelHolder> CreateModelHolder(string owner, string repo, string modelType);
+        Task<IPredictor> GetPredictor(string owner, string repo, string modelType);
+    }
+    public class ModelHolderFactory : IModelHolderFactory


Please split into separate files. In general, .NET has a strong one-class-per-file norm that we'll want to stick with unless there's a type that can't stand on its own. In that case, we'd want to nest it in its parent.

jsquire · 2026-01-09T20:23:05Z

tools/issue-labeler/src/IssueLabeler.Shared/IModelHolderFactory.cs

+        private readonly ConcurrentDictionary<(string, string, string), IModelHolder> models = new ConcurrentDictionary<(string, string, string), IModelHolder>();
+        private readonly ILogger<ModelHolderFactory> logger;
+        private readonly IRepositoryConfigurationProvider configurationProvider;
+        private SemaphoreSlim sem = new SemaphoreSlim(1,1);


+1. This is a good suggestion.

tools/issue-labeler/src/IssueLabeler.Shared/IModelHolderFactory.cs

jsquire · 2026-01-09T20:31:56Z

tools/issue-labeler/src/IssueLabeler.Shared/IModelHolderFactory.cs

+    }
+    public class ModelHolderFactory : IModelHolderFactory
+    {
+        private readonly ConcurrentDictionary<(string, string, string), IModelHolder> models = new ConcurrentDictionary<(string, string, string), IModelHolder>();


Just a quick mention that if you wanted to avoid the "Look at mah Java!" feel, you can shorten up the declarations on either side. Nothing wrong with the verbose form if that's your preference.

private readonly ConcurrentDictionary<(string, string, string), IModelHolder> models = new();

private readonly var models = new ConcurrentDictionary<(string, string, string), IModelHolder>();

jsquire · 2026-01-09T21:39:13Z

tools/issue-labeler/src/IssueLabelerMLPipeline/NuGet.config

@@ -0,0 +1,7 @@
+<?xml version="1.0" encoding="utf-8"?>


We need to remove this. Microsoft has a new policy that requires us to pass through a proxy for all NuGet pulls and this resets that.

jsquire · 2026-01-09T21:58:02Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Common/DataFileUtils.cs

+    /// </summary>
+    /// <param name="text">The text to sanitize.</param>
+    /// <returns>The sanitized text.</returns>
+    public static string SanitizeText(string text)


I think we can add .AsSpan() on text and save a bunch of allocations.

jsquire · 2026-01-09T22:08:47Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Common/LabelType.cs

@@ -0,0 +1,5 @@
+public enum LabelType
+{
+    Category = 1,


Why are we explicitly numbering? Generally, you don't want to do that unless you're actually using the numeric values. We also have an issue here as default(LabelType) ends up being an invalid because it gets initialized as (LabelType)0. We'll want to add an Unknown = 0 at least here.

jsquire · 2026-01-09T22:09:13Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Common/LabelType.cs

+public enum LabelType
+{
+    Category = 1,
+    Service = 2
+}


+1 . We do need that.

jsquire · 2026-01-09T22:11:10Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/DevOpsModelRetrainer/Args.cs

+
+public struct Args
+{
+    public string? GitHubToken { get; set; }


This looks like a file that we borrowed. We'll want to update to our header. I was going to suggest using System.CommandLine but if we inherited this and we know it works, better to not.

jsquire · 2026-01-09T22:13:05Z

GitHub is being really, really laggy today. I left some thoughts but didn't dig into the back half of the files yet.

radhgupta · 2026-01-14T18:42:38Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+            var blobClient = containerClient.GetBlobClient(blobName);
+
+            using var fileStream = File.OpenRead(filePath);
+            await blobClient.UploadAsync(fileStream, overwrite: true);


I noticed there's some blob storage code that's getting repeated across a few files. We've got similar patterns in:

OpenAiIssueGenerator.cs (uploading generated issues)
DevOpsModelRetrainer.cs (uploading models + downloading synthetic data)
IssueTriageContentRetrieval.cs (uploading JSON content + soft delete config)

I was wondering if it we could create a service class for blob related operating and reuse it?

radhgupta · 2026-01-14T18:50:44Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+            _logger.LogInformation($"Uploaded TSV file to blob: {blobName}");
+        }
+
+        private string GetCategoryLabelsForPrompt(IEnumerable<Label> labels)


Seems like GetCategoryLabelsForPrompt and GetServiceLabelsForPrompt are almost similar can we make a single method??

…y.cs Co-authored-by: Jesse Squire <jesse.squire@gmail.com>

radhgupta · 2026-01-14T19:01:52Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+
+            // Download the blob content
+            var response = await blobClient.DownloadContentAsync();
+            var labelsJson = response.Value.Content.ToString();


For large label files, consider streaming the JSON directly instead of loading the entire content into memory.

Suggested change

var labelsJson = response.Value.Content.ToString();

using var stream = await blobClient.OpenReadAsync();

var labels = await JsonSerializer.DeserializeAsync<IEnumerable<Label>>(stream);

radhgupta · 2026-01-14T19:54:21Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+            var response = await _ragService.SendMessageQnaAsync(instructions, message, modelName, contextBlock);
+            var issues = JsonConvert.DeserializeObject<IEnumerable<IssueTriageContent>>(response);
+
+            if (issues == null)


I'm not sure, but is it possible that the AI response could be empty or not have a valid JSON structure? Currently we only check if issues is null after deserialization, but what if the AI returns an empty array, malformed JSON, or unexpected schema?

radhgupta · 2026-01-14T19:57:01Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+            await using (StreamWriter writer = new StreamWriter(fileName))
+            {
+                await writer.WriteLineAsync(FormatIssueRecord("CategoryLabel", "ServiceLabel", "Title", "Body"));
+                foreach (var issue in issues)


for var issues the multiple enumeration isn't too severe since we're deserializing from a string, I think we could materialize the collection once with .ToList() to avoid it

radhgupta · 2026-01-14T19:59:12Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+        private async Task<IEnumerable<Label>> GetLabelsAsync(string repositoryName)
+        {
+            // Initialize BlobServiceClient
+            var containerClient = _blobClient.GetBlobContainerClient("labels");


Consider moving "labels" to configuration or constants class for better maintainability

radhgupta · 2026-01-14T19:59:26Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+
+        private async Task UploadToBlobAsync(string filePath, string repositoryName)
+        {
+            var containerClient = _blobClient.GetBlobContainerClient("generated-issues");


Consider moving container names to configuration or constants class for better maintainability

radhgupta

Overall, it looks good to me :)

radhgupta · 2026-01-14T20:03:14Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+            return string.Join(", ", serviceLabels);
+        }
+
+                public static string FormatTemplate(string template, Dictionary<string, string> replacements, ILogger logger)


I noticed there are static utility methods mixed with instance methods in OpenAiIssueGenerator. The static methods like FormatTemplate, FormatIssueRecord, and SanitizeText break the dependency injection pattern used elsewhere in the class.

Consider either:

Moving these to a dedicated ITextProcessingService that can be injected

Making them instance methods so they can use the existing _logger field

This would make the code more consistent with the DI pattern used throughout the rest of the class

radhgupta · 2026-01-14T20:11:17Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Common/DataFileUtils.cs

+    /// </summary>
+    /// <param name="text">The text to sanitize.</param>
+    /// <returns>The sanitized text.</returns>
+    public static string SanitizeText(string text)


We have identical SanitizeText and FormatIssueRecord methods in both projects. Consider moving these to the shared IssueLabeler.Shared project

radhgupta · 2026-01-14T20:13:01Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/DevOpsModelRetrainer/DevOpsModelRetrainer.cs

+        }
+
+        Console.WriteLine($"Downloading: {blobItem.Name}");
+        await blobClient.DownloadToAsync(localFilePath);


Sequential blob downloads will be slow for large datasets.Can we parallelize it??

tools/issue-labeler/src/IssueLabelerMLPipeline/src/DevOpsModelRetrainer/DevOpsModelRetrainer.cs

radhgupta · 2026-01-14T20:20:18Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/Common/App.cs

+        }
+        catch (AggregateException ex)
+        {
+            Console.WriteLine($"ERROR: Exception occurred: {ex.Message}");


I notice there are quite a few Console.WriteLine statements throughout. Since the rest of the codebase uses ILogger for structured logging, should we consider refactoring these to use a logger instead? This would give us better control over log levels, structured data, and consistency with the rest of the application.

radhgupta · 2026-01-14T20:26:54Z

tools/issue-labeler/src/IssueLabelerMLPipeline/src/DevOpsModelRetrainer/DevOpsModelRetrainer.cs

+    }
+    catch (Exception ex)
+    {
+        Console.WriteLine($"ERROR: Error retraining {typeName} models for {fullRepoName}: {ex.Message}");


Will it be worth adding retry logic like we have in GithubApi.cs??

radhgupta · 2026-01-14T20:33:50Z

tools/issue-labeler/src/IssueLabelerService/IssueGenerator/OpenAiIssueGenerator.cs

+            _ragService = ragService;
+            _logger = logger;
+            _blobClient = blobClient;
+            _retrievalClient = new KnowledgeAgentRetrievalClient(


I notice the KnowledgeAgentRetrievalClient is created in the constructor but never disposed. Since this likely implements IDisposable and holds network resources, it could cause connection leaks in long-running scenarios.

Should ww Make OpenAiIssueGenerator implement IDisposable and dispose the client,

…t/GitHubApi.cs Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>

Linh Phan and others added 21 commits November 4, 2025 15:01

make issue generator

6353aca

Create issue generator

09b62b8

clean up and change how synthetic issues are written out

999b41a

move where file write out is

416b29b

Make new models work for python SDK

b7d961a

Clean up and adding new ML pipeline

cc633eb

Merge branch 'main' of https://github.com/MaiLinhP/azure-sdk-tools

755df2d

Clean up of Legacy Labeler and fully replacing it with new ML pipeline

0297510

More clean up of unused files. Make sure new IModelHolderFactoryLite …

adc12ca

…works with repo specific Configuration

Merge branch 'Azure:main' into main

965124f

Refactoring, changing names, get rid of github actions dependencies

4733eb9

Merge branch 'main' of https://github.com/MaiLinhP/azure-sdk-tools

e60d531

Make issue generator more versatile, and upload result generated issu…

48f324d

…es to blob storage

Fix styling, redo unnecessary changes

3134d09

add EndProject on sln file

f5885d6

fix

e8955f8

Add functionally to train with synthetic data, minor fix

f2d7b76

Refactoring, giving more flexibility on uploading to blob

388bbd4

Merge branch 'Azure:main' into main

af63bb7

delete file

5157131

fix error

4e7cffc

Copilot AI review requested due to automatic review settings December 10, 2025 22:40

MaiLinhP requested review from jeo02, jsquire and radhgupta as code owners December 10, 2025 22:40

Copilot started reviewing on behalf of MaiLinhP December 10, 2025 22:41 View session

Copilot AI reviewed Dec 10, 2025

View reviewed changes

jeo02 requested changes Jan 8, 2026

View reviewed changes

jsquire reviewed Jan 9, 2026

View reviewed changes

radhgupta reviewed Jan 14, 2026

View reviewed changes

Update tools/issue-labeler/src/IssueLabeler.Shared/IModelHolderFactor…

975fafd

…y.cs Co-authored-by: Jesse Squire <jesse.squire@gmail.com>

radhgupta reviewed Jan 14, 2026

View reviewed changes

Update tools/issue-labeler/src/IssueLabelerMLPipeline/src/GitHubClien…

6366a09

…t/GitHubApi.cs Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com>

		// Licensed to the .NET Foundation under one or more agreements.
		// The .NET Foundation licenses this file to you under the MIT license.

		@@ -0,0 +1,55 @@
		// Licensed to the .NET Foundation under one or more agreements.

		@@ -0,0 +1,193 @@
		// Licensed to the .NET Foundation under one or more agreements.

		@@ -0,0 +1,236 @@
		// Licensed to the .NET Foundation under one or more agreements.

		@@ -0,0 +1,43 @@
		// Licensed to the .NET Foundation under one or more agreements.

-            var labelsJson = response.Value.Content.ToString();
+     using var stream = await blobClient.OpenReadAsync();
+    var labels = await JsonSerializer.DeserializeAsync<IEnumerable<Label>>(stream);

Conversation

MaiLinhP commented Dec 10, 2025

Added an AzureSdkIssueGeneratorService Function App that will reference azure-sdk-for-net issues to generate synthetic issues for other repos and add TSV file to blob storage for model training if needed.

Replaced all old ML labeler processes with streamlined IssueLabelerMLPipeline. Most of this pipeline is meant to be run locally for more control over model training processes. It includes:

Unified config setup.

Simplified the flow from AzureSdkIssueLabelerService to Labeler.

Cleaned up codebase

Uh oh!

microsoft-github-policy-service bot commented Dec 10, 2025

Contribution License Agreement

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

jeo02 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Added an `AzureSdkIssueGeneratorService` Function App that will reference azure-sdk-for-net issues to generate synthetic issues for other repos and add TSV file to blob storage for model training if needed.

Replaced all old ML labeler processes with streamlined `IssueLabelerMLPipeline`. Most of this pipeline is meant to be run locally for more control over model training processes. It includes:

Simplified the flow from `AzureSdkIssueLabelerService` to `Labeler`.